Under-Approximating Expected Total Rewards in POMDPs

Abstract We consider the problem: is optimal expected total reward to reach a goal state in partially observable Markov decision process (POMDP) below given threshold? tackle this—generally undecidable—problem by computing under-approximations on these rewards. This done abstracting finite unfoldings of infinite belief MDP POMDP. The key issue find suitable under-approximation value function. provide two techniques: simple (cut-off) technique that uses good policy POMDP, and more advanced (belief clipping) minimal shifts probabilities between beliefs. use mixed-integer linear programming (MILP) such probability experimentally show our techniques scale quite well while providing tight lower bounds reward.